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Why GAO Did This Study 

The Coast Guard is developing a 
Web-based information system to 
replace an aging computer 
system that it uses to track safety 
and law-enforcement actions — 
such as inspections, drug 
interdiction, and oil spill 
assistance — involving 
commercial and recreational 
vessels. In 1995, the Coast Guard 
awarded a contract to develop 
the new system, called MISLE. 
After the project encountered 
cost and schedule problems, 
development responsibility was 
transferred to the Coast Guard's 
systems development center in 
1999, and the contract terminated 
after about $26 million had been 
spent. 

The Coast Guard's history of 
systems development problems 
and information technology 
weaknesses prompted the 
Subcommittee to ask GAO to 
review MISLE's current status 
and risks. 



What GAO Recommends 

GAO is making several 
recommendations in the four key 
areas of system requirements, 
software testing, transition 
planning, and risk mitigation. 

In comments on a draft of our 
report, Coast Guard officials 
generally agreed with GAO's 
recommendations, but noted that 
some of the risks were "tradeoff 
decisions," necessary to deliver an 
initial version of MISLE. 



What GAO Found 

The Coast Guard has made progress since taking over its Marine 
Information for Safety and Law Enforcement (MISLE) system 
development; it is now poised to deploy the system with limited 
capabilities. But much remains to be accomplished in order to deliver the 
complete system, with only about $11 million of its $61 million estimate 
remaining. 

MISLE is facing risks in four key areas: 

• Changing system requirements. MISLE requirements have 
repeatedly changed; several have been dropped, and others 
postponed to later development phases due to problems found during 
testing and the Coast Guard's emphasis on replacing its current 
system as soon as possible. Other requirements are being added or 
accelerated. Such a continually changing scenario increases the risk 
that MISLE will fall short of user needs. 

• Software testing. The Coast Guard undertook risky software 
testing practices in that it deferred testing some functions and did not 
resolve all critical problems uncovered before moving on to the next 
testing stage. This approach increases the likelihood that the system 
will not perform as expected and/or may take longer to develop than 
anticipated. 

• Transition planning. Deployment of MISLE involves planning for 
accurately moving data from the older system and training system 
users. However, critical transition plans are not yet complete. 
Beyond adding to possible delays, the absence of transition plans and 
insufficient training increase the chance of user discomfort with the 
new system. 

• Risk management. Finally, the Coast Guard's risk management 
approach has been ineffective: risks were not assigned severity 
ratings, and not all have been prioritized. Further, the Coast Guard 
has not developed detailed mitigation plans for all significant risks. 

Unless these challenges are successfully addressed, performance 
shortcomings, cost escalation, and schedule delay are likely. 
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October 17, 2001 



The Honorable Frank LoBiondo 
Chairman 

The Honorable Corrine Brown 
Ranking Minority Member 

Subcommittee on Coast Guard and Maritime Transportation 
Committee on Transportation and Infrastructure 
House of Representatives 

Over the last decade, the United States Coast Guard (USCG) has 
experienced difficulties in acquiring the Marine Information for Safety and 
Law Enforcement (MISLE) system — an information system to track marine 
safety and law-enforcement activities involving commercial and 
recreational vessels. In 1999, the Coast Guard terminated a contract to 
acquire MISLE, after spending about 4 years and $26 million, and is instead 
developing the system at its Operations Systems Center. As you requested, 
our objective was to provide an update on MISLE's status, plans, and 
technical and programmatic risks. 

To fulfill this objective, we evaluated MISLE project plans, costs, and 
schedules by comparing original and current job management documents. 
We also assessed technical and programmatic risks facing the MISLE 
acquisition and USCG's plans for addressing those risks. We performed our 
work from May through September 2001, in accordance with generally 
accepted government auditing standards. USCG officials provided us with 
comments on a draft of this report; they are discussed in the "Agency 
Comments" section. 

On September 18, 2001, we provided a detailed briefing to your office on 
the results of this work. The briefing slides are included in appendix I. The 
purpose of this letter is to provide the published briefing slides to you and 
to officially transmit our recommendations to the Secretary of 
Transportation. 

In brief, we reported that USCG has made progress in developing MISLE 
and was poised to deploy a minimum level of functionality in November 
2001. However, the system was several months behind schedule, and 
USCG had already spent most of the $61 million acquisition cost estimate — 
most of it on the original contract. With only about $11 million remaining, 
much remains to be done to deliver the complete system. In its efforts to 
develop and deploy a complete MISLE system, USCG faces significant 
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challenges and risks in several areas, including managing system 
requirements and user expectations, testing the system, transitioning to an 
operational system, and managing program risks. We made specific 
recommendations to address these risks. 



Recommendations ^° mitigate USCG's MISLE risks, we recommend that the Secretary of 

Transportation direct the USCG Commandant to ensure that the 
appropriate officials complete the following actions. 

In the system requirements area, 

• define and prioritize, in conjunction with system users, all needed 
system functions, corrections, and enhancements that must occur to 
meet valid user needs; and 

• develop cost and schedule estimates for providing these functions, 
corrections, and enhancements. 

In the software testing area, 

• close all critical problems before initiating the next state of testing. 
In the area of transition planning, 

• finalize and implement Vessel Documentation System (VDS) transition 
plans, and 

• develop and implement VDS training materials. 
In the risk mitigation area, 

• develop a single list of system risks, 

• evaluate system risks to determine their severity and prioritize these 
risks, 

• develop and implement comprehensive mitigation strategies for each of 
the risks, and 

• regularly oversee the status of risks and risk mitigation efforts to 
determine whether additional mitigation activities are warranted. 
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AgGllCy Comments ^ e P rov ided drafts of our briefing and this report to Department of 

Transportation and USCG officials, including representatives of the Office 
of the Secretary of Transportation and the USCG Acquisition Management 
Office. We met with USCG officials to obtain their comments on our drafts. 
These officials generally agreed with our recommendations, but 
characterized the changing requirements and testing issues we raised as 
"tradeoff decisions," necessary to deliver an initial version of MISLE. 
USCG officials also noted that, given the recent terrorist attacks, MISLE 
deployment will likely be delayed until at least December 2001. 



We are sending copies of this report to the Secretary of Transportation, the 
USCG Commandant, the Director of the Office of Management and Budget, 
and other interested parties. Copies will also be made available to others 
upon request. 

Should you or your staffs have any questions concerning this report, please 
contact me at (202) 512-6240 or by e-mail at koontzl@gao.gov Nabajyoti 
Barkakati, Barbara Collier, Michael Fruitman, Colleen Phillips, Margaret 
Sullivan, and Glenda Wright were major contributors to this report. 

Linda D. Koontz 

Director, Information Management Issues 
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GAO's September 18, 2001, Briefing 
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nH^^™ Accountability * Integrity * Reliability w 


• Objective, Scope, and Methodology 




• MISLE Background 




• MISLE Status and Plans 




• Key Risk Areas 




• System requirements 




• Software testing 




• Transition planning 




• Risk management 




• Conclusions 




• Recommendations 




• Agency Comments and Our Evaluation 
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Objective 

To provide an update on MISLE's status, plans, and technical and 
programmatic risks 



Scope and Methodology 

We evaluated MISLE project plans, costs, and schedules by comparing 
original and current job management documents. We also assessed technical 
and programmatic risks facing the MISLE acquisition, as well as USCG's 
plans for addressing those risks, by evaluating USCG program documents, 
comparing them to accepted system engineering principles, and interviewing 
project officials, developers, and system users. Further, we reviewed plans 
for future MISLE enhancements and discussed these enhancements with 
system users and developers. 

We conducted our review at USCG headquarters in Washington, D.C., and at 
USCG's Operations Systems Center and National Vessel Documentation 
Center in the Martinsburg, WV, area. We conducted our work from May 
through September 2001 , in accordance with generally accepted government 
auditing standards. 
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Overview 



MISLE is intended to be a Web-based information system for entering and 
obtaining data on Coast Guard activities concerning commercial and 
federally documented recreational vessels in support of USCG's marine 
safety and law enforcement missions. 

A software-intensive system, the MISLE project also includes the purchase 
of hardware, including network servers and data storage devices. 

MISLE is to support over 5000 system users at about 550 sites using the 
existing Coast Guard intranet. 
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In the mid-1990s, MISLE was envisioned to develop and integrate two major 
systems — the Marine Safety Network and the Vessel Identification and 
Documentation System — and to integrate them with a third, already developed 
system, the Law Enforcement Information System II. 

Since then, however, this concept has evolved. A description of the 
subsystems and recent changes to these subsystems are discussed below: 

• Marine Safety Network (MSN) 

a system to allow Coast Guard personnel to input and obtain information 
on USCG marine safety activities, such as vessel inspections and 
boardings; it relies on vessel data from the Vessel Documentation System 
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MISLE components (cont'd) 

• Vessel Identification and Documentation System 

a system planned to integrate two subsystems in response to Public Law 
100-710 (commonly called the Ship Mortgage Act of 1988): 

• Vessel Identification System — planned to automate and integrate 
vessel registration information from participating states; due to 
complications in implementing this system, USCG no longer plans to 
provide VIS functionality as part of MISLE (see Key Risk Area — System 
Requirements) 

• Vessel Documentation System (VDS) — a system to support the 
National Vessel Documentation Center's processes for documenting 
vessels 



• Law Enforcement Information System II (LEIS-II) 

a system for tracking USCG law enforcement activities, such as drug 
interdiction activities; due to the age of the LEIS-II platform, USCG now 
plans to replace LEIS-II rather than integrate it with MISLE 
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As of early September 2001 , USCG's estimates for MISLE costs 
and deployment schedules included the following: 




• Acquisition cost: $61 million through 2003 




• Life cycle cost: $94 million through 2008 




• Initial capability: November 2001 




• Final capability: September 2003 
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A Q ^ Q MISLE Background 
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In 1986, USCG began developing requirements for replacing its 
legacy Marine batety Information bystem (Mblb) by 1995. 




In the early 1990s, USCG delayed plans to replace MSIS in order to 
integrate requirements for multiple systems into one system 
development effort — called MISLE. 




In 1995, USCG awarded a contract to Computer Sciences 
Corporation (CSC) to develop and deliver a complete MISLE system 
by 2002. At that time, USCG officials estimated that this contract 
could cost up to $35 million. 




• A March 1999 change required the developer to replace MSIS 
functions by June 2001 . 
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A Q- ^ Q MISLE Background 
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The MISLE development contract encountered escalating costs and 
schedule delays; it was "partially terminated for government convenience" 
in October 1999. 



• CSC delivered two components under this contract: 

• Vessel Identification System — a system to automate and 
integrate vessel registration information from participating states. 
This system was never implemented due to problems in 
integrating state data. USCG no longer plans to provide VIS 
functionality as part of MISLE. 

• Mission Analysis and Planning — a decision support tool for 
obtaining statistical information from the legacy MSIS system 



• Total spent on the CSC contract: $26 million 



In October 1999, USCG transferred responsibility for MISLE development 
from CSC to its Operations Systems Center (OSC) — a government-owned, 
contractor-operated facility. USCG's acquisition office manages the 
project. 

9 
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OSC's strategy is to develop and deploy a minimal level of MISLE 
functionality as soon as possible to replace MSIS. 

• MSIS — once scheduled for termination in 1995 — is still serving as the 
primary information system for marine safety programs. 

• MSIS has failed repeatedly over the past several years, causing days 
and weeks of data processing backlogs. 

• USCG officials stated that MSIS hardware is no longer supportable; 
the original vendor is no longer in business, and the agency is 
currently reusing parts to keep the system running. 
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^ Accountability integrity* Reliability Acquisition Strategy (cont'd) 



OSC's system development approach: 

• Rapid Application Development/Joint Application Design 

a system development methodology that focuses on involving users 
early and often in the design and development of a system in order to 
develop it quickly. 



• Timebox Methodology 

a system development methodology, often used in conjunction with 
the Rapid Application Development methodology, in which a time 
limit is established for developing a system (this forces users to focus 
on a minimal set of requirements and allows them to drop some 
functions if necessary to meet the time limit). 
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MISLE Background 
Planned Delivery Schedule 



OSC's three-phased delivery approach: 



MISLE Phase 1 

Oct. 1999 to Nov. 2001 

Deliverables: 

• initial MSN capability needed to replace MSIS 

• initial VDS capability 



MISLE Phase 2 

Feb. 2001 to March 2002 
Deliverables: 

• replacement of LEIS-II 

• extended MSN and VDS capabilities 

• initial data warehouse 





MISLE Phase 3 




Feb. 2002 to Sept. 2003 




Deliverables: 




• completion of remaining MSN and VDS capabilities 




• expanded data warehouse 
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Accountability * Integrity * Reliability PfOgTSSS tO DfltS 



MISLE Phase 1 

40 high level requirements have been translated to over 13,000 detailed 
software requirements supporting two key components: the Marine Safety 
Network (MSN) and the Vessel Documentation System (VDS) 



MSN Status: 

• Completed system testing 1 

• User input obtained throughout 



VDS Status: 

• Major components postponed to later releases 

• Completed system testing 

MISLE Phase 2 

Development began February 2001 



1 USCG tests increasingly larger portions of MISLE in stages: (1) unit testing-informal tests of software modules; (2) functional testing-formal tests of 
software modules; (3) integration testing-testing the integration of all modules in a core area; and (4) system testing-testing all of the core areas in a 
deliverable. MISLE then undergoes operational test and evaluation (OT&E)-testing of operational effectiveness and suitability by system users. 
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Costs 



USCG's $61 million total estimated MISLE acquisition cost includes: 



• about $7 million spent on requirements analysis, design, and 
validation in the early 1990s 



• $26 million spent on the 1995 MISLE contract that was terminated 



• about $28 million for current MISLE development efforts, including 
• almost $17 million spent through fiscal year 2001 to develop 

• most of MISLE Phase 1 

• part of MISLE Phase 2 



• about $1 1 million in fiscal year 2002 and beyond 

• to complete residual tasks supporting MISLE Phases 1 and 2 

• to develop and implement MISLE Phase 3 
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Schedule 



MISLE Phase 1 has fallen behind schedule. 



MISLE Phase 1 
milestones 


Dec. 1999 

baseline 

change 3 


June and 
Dec. 2000 
estimates" 


May 2001 
estimate 


Aug. 1, 2001 
estimate 


Sept. 5, 2001 
estimate 


System testing complete 


N/P 


N/l 


7/20/01 


MSN 8/15/01 
VDS 9/15/01 


MSN 8/27/01 
VDS 8/27/01 


Operational Test and 
Evaluation complete 


6/01 


2/01 


8/15/01 


MSN 8/30/01 
VDS 9/30/01 


MSN 9/21/01 
VDS 9/21/01 


MISLE month complete 


N/l 


N/l 


9/15/01 


9/30/01 


10/19/01 


Data migration complete 


N/l 


N/l 


9/30/01 


10/10/01 


11/2/01 


MISLE Phase 1 deployed 


N/l 


4/01 


10/01 


10/15/01 


11/5/01 



a ln commenting on a draft of this briefing, the MISLE project manager noted this baseline document also includes 
an OT&E completion range from January to September 2001 , and that OT&E would be completed within this 
range. However, USCG internal management reports reflect the June 2001 completion date. In fact, these reports 
show MISLE in breach of its schedule. 

"These estimates are identified in the June 2000 MISLE Implementation Plan, as well as the December 2000 
MISLE Acquisition and Project Management Plans. In commenting on a draft of this briefing, USCG officials 
characterized these documents as internal, optimistic project plans--not firm schedule estimates. 
"Not identified. 
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A Q ^ Q MISLE Status and Plans 

Accountability * Integrity * Reliability SChGdlll6 (COtlt d) 

USCG officials identified the following reasons for schedule delays: 

• Problems in integrating VDS and MSN components caused a 
major VDS redesign (October 2000) 

• USCG needed to retrofit MSN to work with the current version of 
Internet Explorer (January 2001) 

• VDS functional testing was delayed because of system instability 
(January 2001) 

• MSN integration testing identified a missing function (June 2001): 

• ability to communicate with USCG's financial center 

• VDS integration testing encountered problems (July 2001): 

• test scripts would not run at users' facility 

• shortcoming in ability to edit MSIS data migrated to MISLE 
(estimated 4 week delay) 
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The MISLE acquisition is facing considerable challenges in the following 
key risk areas: 



System Requirements 
Software Testing 
Transition Planning 
Risk Management 
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L GAO Key Risk Areas 

Accountability * Integrity * Reliability SyStSITI RGCjUjrGITISntS 



System requirements define the minimum functions and performance 
levels needed to satisfy user needs. USCG identified system 
requirements for MISLE in 1994. Also, when the agency restructured the 
MISLE program in 1999, it identified specific functions to be delivered in 
each of the three MISLE phases. 



However, MISLE requirements and the functions to be provided in each 
of the three deliverables are now changing. USCG plans to drop several 
of MISLE's original system requirements and has recently postponed key 
functions due to problems encountered during testing and the agency's 
need to field a system quickly. Also, in response to system users' needs, 
USCG plans to add and accelerate key functions. Further, while testing 
and training on the system, users identified new and missing functions 
which USCG plans to add to MISLE later. 



USCG's plans for dropping, delaying, accelerating, and adding functions 
are discussed below. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



USCG is planning to drop significant MISLE requirements: 

• Vessel Identification System — This system, planned to automate 
states' vessel registration data and integrate it with data on federally 
documented vessels in response to the Ship Mortgage Act of 1988, 
was developed as part of MISLE but never successfully 
implemented. USCG is considering developing a new system outside 
MISLE. 

• Portable/Remote Access — This function was planned to provide the 
ability to access MISLE information from remote locations, such as 
Coast Guard cutters. USCG officials stated that MISLE will allow for 
remote access via laptop and dial-up modem, but that providing 
cutter connectivity will require decisions on the USCG infrastructure 
that are beyond MISLE's scope. USCG is considering leasing 
satellite time to provide cutter connectivity. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



Dropped requirements (cont'd) 

• External Access — This function was planned to provide non-USCG 
personnel with access to MISLE data. USCG officials stated that MISLE 
has this capability but will not provide external access because of 
security concerns. USCG plans to reassess the need for external 
access on a case-by-case basis. 



• Merchant Mariner Licensing and Documentation — This function was 
to provide licensing and documentation capabilities within MISLE. 
However, instead of waiting for MISLE, USCG decided to develop a 
stand-alone system to provide this capability. MISLE Phase 1 is 
planned to link to the Merchant Mariner Licensing and Documentation 
stand-alone system. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



Dropped requirements (cont'd) 



• Pollution Funds Management — This function was to provide the 
ability to manage pollution funds within MISLE. Instead, USCG 
developed an alternative system outside MISLE to provide this 
functionality. MISLE will provide a link to that system. 



• Personnel Resource Management — This function was to provide the 
ability to manage personnel within MISLE. Instead, USCG developed 
an alternative system outside MISLE to provide this functionality. USCG 
officials expect that MISLE will be able to access data in that system. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



USCG has delayed some MISLE functions: 



Significant Phase 1 deliverables are now planned for later releases. 
Examples include VDS' ability to 

• provide Abstracts of Titles 

• provide Certificates of Ownership 

• automate N VDC's paper-intensive work processes 



Key Phase 2 deliverables are now planned to be Phase 3 deliverables. 
Two high priority examples include 

• VDS' ability to handle requests for nonvessel information (this 
involves non-U. S. corporations and oil spill response 
organizations) 

• MSN's ability to capture information on the approval and oversight 
of the construction of portable quarters on vessels 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



Delayed functions (cont'd) 



Other Phase 2 deliverables are being reconsidered — that is, USCG 
reported that they may not be "doable" in Phase 3. Examples include 
MSN's ability to 



• submit Web forms, such as vessel arrival, inspection request, and 
marine event application forms 

• capture and track information on reviews of manuals (such as 
Operations Manuals, Emergency Evacuation Plans, and 
Passenger Terminal Security Plans) 



Other examples include VDS' ability to 



• provide electronic access to data in response to FOIA requests 

• allow e-commerce, such as credit card transactions and 
submission of Web forms 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



USCG has added and accelerated MISLE functions: 



In response to user needs, USCG has decided to 



• replace LEIS-II in Phase 2, rather than integrate it with MISLE 

• accelerate a critical function, incident response planning, from 
Phase 3 to Phase 2 



In commenting on a draft of this briefing, USCG officials noted that the 
agency has also moved multiple functions from Phase 2 to later releases 
of Phase 1 in response to user requests to get these functions sooner. 
However, in prior discussions, USCG officials acknowledged that they 
have not yet scheduled these later releases and, in fact, some of the later 
Phase 1 releases may not be delivered until after Phase 2 is delivered. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



USCG plans to add functions to MISLE 

While testing and training on MISLE, system testers and users are 
identifying both needed system corrections and new user needs that 
MISLE does not currently provide. USCG plans to add these functions to 
MISLE as "future enhancements." 



• As of August, USCG identified 72 future enhancements. Examples 
include 

• making vessel details on maps readable 

• incorporating data from additional information systems outside 
MISLE 

• making a print icon work 

In commenting on a draft of this briefing, MISLE project officials stated 
that these future enhancements are low-level requirements and user 
preferences, and that they do not jeopardize the project's ability to deliver 
on its agreed-upon scope. 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability System Requirements (cont'd) 



USCG has developed a schedule for the functions that it deems are the 
highest priority — MSN release 1.1 and 1.2, and MISLE Phase 2. 
However, USCG has not yet prioritized all of the additional and delayed 
MISLE functions, nor has it developed schedules for providing them. For 
example, USCG has not yet established a schedule for delivering the 
significant VDS functions that were deferred from Phase 1 to the VDS 
release 1 .1 , or for the many other MSN and VDS functions anticipated in 
later releases of Phase 1 . 



By dropping and/or delaying key MISLE functions, USCG runs the risk 
that the system will not function as intended or expected, and that the 
deployed system will fall short of user needs. Further, without a schedule 
or cost estimate for the delayed functions and enhancements, it is not 
clear when these functions will be implemented nor how much it will cost 
to implement them. 
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According to leading information technology organizations, to be effective, 
software testing practices should be planned and conducted in a structured 
and disciplined fashion. Typically, this involves testing increasingly larger 
increments of a system until the complete system is tested and accepted, 
and resolving critical problems before moving to the next phase of testing. 



USCG policies call for 

• testing MISLE software in successively larger increments 2 

• unit testing 

• functional testing 

• integration testing 

• system testing 

• operational test and evaluation (OT&E) 

• managing problems reported during testing: 

• problem reports are to be prioritized in levels 1 to 5 (levels 1 and 2 
are most critical), fixed, tested, and closed 

• all critical problem reports (levels 1 and 2) are to be closed before the 
next stage of testing begins 

2 We define similar concepts in our guide for managing testing activities, 27 
Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-1 0.1 .21 , November 1 998). 
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Accountability integrity* Reliability Software TeStjng (COtlt'd) 



USCG's current testing schedule for MISLE components: 



MSN Integration 
Testing 

MSN System 
Testing 

MSN OT&E 



VDS Integration 
Testing 

VDS System 
Testing 

VDS OT&E 



(4/1/01) 



(6/26/01) 



( 6/30/01) 



(5/21/01) 



(8/27/01) 

(9/10 /01) (9/2 1/01) 

J8/10/01) 
(8/16/01) (8/27/01) 

(9/10/01) (9/21/01) 



April May 



June 

2001 



July Aug 



Sept 
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USCG has undertaken risky testing practices: 



• Key VDS functions were not tested during functional testing because 
they were under development at the time and were deferred to 
integration testing. For example: 



• system security 

• processing and printing Abstracts of Titles 

• printing Certificates of Documentation 
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^ Accountability integrity* Reliability Software TeStjng (COtlt'd) 



Risky testing practices (cont'd): 

• Significant problems were not closed before the next stage of testing 
began. For example: 

• 9 critical problems identified during MSN integration testing were 
not closed before system testing began 



USCG's testing practices increase the risk that MISLE will not perform as 
expected, or may take longer to develop than expected. 
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According to the CIO Council's guidance on enterprise architectures, 
agencies should carefully plan the transition from legacy systems to new 
systems. To ease the transition from MSIS to MISLE, USCG developed a 
transition plan that calls for developing data migration plans and training 
system users. 



USCG has acknowledged that transitioning from MSIS to MISLE will be 
difficult. It will involve moving data from one system to the next, ensuring 
the accuracy of the transported data, and training users on new business 
processes. USCG has undertaken several transition efforts: 



• MSN data migration plan has been developed 

• Hundreds of MSN users have undergone training 



• A 4-week period, known as MISLE month, is planned in which MSIS 
and MISLE will be run concurrently to allow MSN users to become 
acquainted with the new system and new work processes 
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A Q ^ Q Key Risk Areas 

Accountability integrity* Reliability Transition Planning (cont'd) 



Despite its transition guidance, USCG has not yet completed critical VDS 
transition planning because the initial system deliverables changed 
dramatically in July 2001 . Specifically, 



• VDS transition plan has not been finalized 

• VDS training plan has not been developed 

• VDS users have not been trained 



VDS users stated that training and transition planning are not critical for 
the first release of VDS, because the functions being delivered will not 
dramatically affect their operations. However, they acknowledge that 
transition and training are critical to the next software release, 3 which is 
planned to reengineer and automate their work processes. 



Incomplete plans for transitioning to a new system increase the likelihood 
that the system will not be deployed on schedule. Insufficient training 
plans also increase the likelihood that users will not readily adapt to new 
work processes and systems. 



3 USCG has not yet scheduled the next software release for VDS. 32 
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Risk management is a key component of a sound systems development 
approach. An effective risk management approach typically includes 
identifying, prioritizing, resolving, and monitoring project risks. In support of 
this approach, USCG's risk management plan calls for 



• assigning a severity rating (high, medium, or low) to risks that bear 
particular attention and placing these risks on a "risk watch list," 

• prioritizing these risks, 

• planning a response or strategy for each risk on the risk watch list, and 
drafting a detailed response plan, and 

• reviewing and evaluating all risks on the the risk watch list during 
monthly management meetings. 
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GA r\ Key Risk Areas 
^JtJ± Ris k Mana g e m en, (cont'd) 



USCG has not effectively implemented its risk management plan. 

• USCG developed three different risk lists, and none of the risks were 
assigned a severity rating. 

• Risks on two of the risk lists have not been prioritized. 

• USCG developed detailed mitigation strategies for some, but not all risks. 
For example, 

• detailed plans exist for data migration and transition risks 

• such plans do not exist for managing user expectations and VDS 
instability risks 

• USCG officials addressed some, but not all, active risks at monthly status 
briefings. Additionally, USCG officials stated that they discuss all active 
risks during monthly management meetings; however, we found no 
evidence that all risks were addressed at these informal meetings. 

Inadequate risk management practices increase the likelihood that MISLE will 
not be delivered on schedule, within budget, and able to perform as expected. 
USCG officials acknowledged areas where they could improve their risk 
management and have undertaken improvement efforts. 
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Conclusions 



After a long history of problems in developing MISLE, USCG ended the 
original MISLE contract in October 1999. Since that time, USCG has made 
progress in developing MISLE and is now poised to deploy a minimum level 
of functionality in November 2001 . However, MISLE is several months 
behind schedule and USCG has already spent most of its $61 million 
acquisition cost estimate-most of it on the original contract. With only about 
$1 1 million remaining, much remains to be done. 

In its efforts to develop and deploy a complete MISLE system, USCG faces 
significant challenges in several areas: 

• managing system requirements and user expectations 

• testing the system 

• transitioning to an operational system, and 

• managing program risks 

Unless USCG can effectively address these challenges, MISLE deployment 
schedules are likely to slip further and costs are likely to increase. Further, 
MISLE may fall short of user expectations and its promised functionality. 
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Recommendations 



To mitigate MISLE risks, we are making several categories of 
recommendations to USCG's MISLE program managers: 



System Requirements 

• Define and prioritize, in conjunction with system users, all needed 
system functions, corrections, and enhancements that must occur to 
meet valid user needs. 

• Develop cost and schedule estimates for providing these functions, 
corrections, and enhancements. 

Software Testing 

• Close all critical problems before initiating the next stage of testing. 
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Accountability * Integrity * Reliability 



Transition Planning 

• Finalize and implement VDS transition plans. 

• Develop and implement VDS training materials. 



Risk Mitigation 

• Develop a single list of system risks. 

• Evaluate system risks to determine their severity and prioritize these 
risks. 

• Develop and implement comprehensive mitigation strategies for each 
of the risks. 

• Regularly oversee the status of risks and risk mitigation efforts to 
determine whether additional mitigation activities are warranted. 
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Agency Comments and our Evaluation 



In providing oral comments on a draft of this briefing, USCG officials 
generally agreed with our recommendations, but disagreed with our 
conclusion that unless the agency can address key risk areas, MISLE 
deployment schedules are likely to slip further, costs are likely to increase, 
and MISLE may fall short of user expectations and its promised 
functionality. 

USCG officials stated that many of the changing requirements and testing 
issues we raised were "tradeoff" decisions made to meet the primary goal 
of delivering an initial version of MISLE to replace MSIS. Additionally, 
USCG officials stated that some MISLE functions were deferred to later 
releases in order to accelerate other user-requested functions. USCG 
officials expressed confidence that they will deliver MISLE within budget 
and on schedule. 
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Agency Comments and our Evaluation 



Our evaluation: We acknowledge that USCG expects to meet its primary 
goal of replacing MSIS in the next few months. However, MISLE will now 
provide fewer capabilities than originally planned, and significant functions 
have been delayed because of problems in testing. 

By dropping and delaying key MISLE functions, USCG runs the risk that the 
system will fall short of user needs. Further, because there are no schedule 
or cost estimates for most of the delayed functions, it is not clear when they 
will be delivered or what they will cost. 
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The General Accounting Office, the investigative arm of Congress, exists to 
support Congress in meeting its constitutional responsibilities and to help improve 
the performance and accountability of the federal government for the American 
people. GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of accountability, 
integrity, and reliability. 
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